Skip to main content

On-Device Integration

DynamoGuard On-Device enables you to run guardrail models on-device, using a hybrid cloud/edge environment. Guardrail models run on-device and make external calls to your secure cloud-hosted LLM.

DynamoGuard Diagram

How it Works​

  • To install DynamoGuard On-Device, a 1.6B base model is installed one-time and LORA adapter weights are used to add custom guardrails as needed. Enterprises can securely add/remove adapter weights as needed through their device admin app.
  • When an application on the device wants to make an external LLM call, it first sends a request to DynamoGuard. The correct adapter for the task is loaded onto the CPU/GPU/NPU, fused with the base model, and then inference is run on the query.
  • If the query is deemed safe, then it is forwarded to the cloud-hosted LLM.
  • The model's response is again analyzed by DynamoGuard and if safe, sent back to the application